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Moment Generating Functions 


6.1.3 Moment Generating Functions 

Here, we will introduce and discuss moment generating functions (MGFs). Moment 
generating functions are useful for several reasons, one of which is their application to 
analysis of sums of random variables. Before discussing MGFs, let's define moments. 


Definition 6.2. The nth moment of a random variable X is defined to be E\X n ~\. The nth central 
moment of X is defined to be E\(X— EX) n \. 


For example, the first moment is the expected value E\X\. The second central moment is the 
variance of X. Similar to mean and variance, other moments give useful infonnation about 
random variables. 

The moment generating function (MGF) of a random variable X is a function M x (s) defined as 

M^s) = E[e sX ]. 

We say that MGF of X exists, if there exists a positive constant a such that M^s) is finite for 
all s £ [ - a, a\. 

Before going any further, let's look at an example. 

Example 6.3 

For each of the following random variables, find the MGF. 
a. X is a discrete random variable, with PMF 


! 3 

2 


k= 1 

k = 2 


b. Tis a Uniform^ 0, 1) random variable. 
• Solution 

o a. For X, we have 
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M&) = E\e sX } 

1 s 1 2s 

= ~e s + 

3 3 

which is well-defined for all s 6 R. 
b. For Y, we can write 

My(s) = E\e sY ] 

= \\/ yd y 

e s - 1 
s 

Note that we always have My( 0) = E[e Q ' Y ] = 1, thus M^s) is also well- 
defined for all s £ R. 


Why is the MGF useful? There are basically two reasons for this. First, the MGF of X gives us 
all moments of X. That is why it is called the moment generating function. Second, the MGF 
(if it exists) uniquely determines the distribution. That is, if two random variables have the 
same MGF, then they must have the same distribution. Thus, if you find the MGF of a random 
variable, you have indeed determined its distribution. We will see that this method is very 
useful when we work on sums of several independent random variables. Let's discuss these in 
detail. 


Finding Moments from MGF: 


Remember the Taylor series for e x : for all x G R, we have 


Now, we can write 


x x 

2 ! + 3 ! 


2 x =1+jc+ — + — + ... 


00 


I 

k=0 


x k 

k\ 


„sX. 


L 

k =o 


(sX) 1 ' 

~k\ 


00 


I 

k=0 


X k s k 

TT 


Thus, we have 
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M J fs) = E[e sX ] = 

k =o *• 

s k 

We conclude that the Ath moment of X is the coefficient of ~ in the Taylor series of Mfs). 
Thus, if we have the Taylor series of M^s), we can obtain all moments of X. 


Example 6.4 

If Y ~ Uniform (0, 1), find E[Y k ] using Mfs). 

• Solution 

o We found My(s) in Example 6.3, so we have 


Mfs) 


e s ~ 1 


00 i 


\k=Q 


UU | 

ly S 1 

s h\ kl 


k -1 


k\ 


-z 

k= 1 
00 

1 1 <3 

= £o 


1 


Thus, the coefficient of 77 in the Taylor series for Mf.s) is 7777 , so 

£[#1 - 777- 


We remember from calculus that the coefficient of 77 in the Taylor series of Mjfs) is obtained 
by taking the Ath derivative of Mfs) and evaluating it at s = 0. Thus, we can write 
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m*] = »-o- 

We can obtain all moments of X^ from its MGF: 


k=0 K - 


E[X k ] 


d k 


Example 6.5 

LetX ~ Exponential^ l). Find the MGF ofX, M X (s), and all of its moments, E[X k ], 
• Solution 

o Recall that the PDF of X is 

fx (*) = A.e~ lx u(x), 


where u(x ) is the unit step function. We conclude 
M^s) = E[e sX ] 

= j^Xe ~ Ax e sx dx 


, - (/l-5)x 


for s < X 


Jo 


X 

= --, for 5 < X. 

X - s 

Therefore, M^s) exists for all s < X. To find the moments ol'X, we can write 
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M ~ Tr s 

1 


5 



We conclude that 


E[X k ] = 


k\ 


for k = 0, 1, 2,... 


Example 6.6 

Let X ~ Poisson(X). Find the MGF ofX, M^s). 

• Solution 

o We have 

k k 

P x (k) = e ~ 1 — , for A: = 0 , 1 , 2 ,... 

Thus, 

M^s) = E[e sX ] 


00 

X e sk e 
k =0 

A:! 

00 

P sk — 

k\ 

k =0 


00 

(Ae 5 )* 

k= 0 

K : 


= e A e /JJ (Taylor series for e x ) 
= e k ^ e 1 \ for all s £ R. 
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As we discussed previously, the MGF uniquely detennines the distribution. This is a very 
useful fact. We will see examples of how we use it shortly. Right now let's state this fact more 
precisely as a theorem. We omit the proof here. 

Theorem 6.1 Consider two random variables X and Y. Suppose that there exists a positive 
constant c such that MGFs of X and Y are finite and identical for all values of s in [ - c, c\. 
Then, 


Fj(i) = Fy(t), for all t 6 R. 


Example 6.7 

For a random variable X, we know that 


2 

M 'x(s) = for 5 G ( - 2, 2). 


Find the distribution ofX. 

• Solution 

o We note that the above MGF is the MGF of an exponential random variable with 
2 = 2 (Example 6.5). Thus, we conclude that A ~ Exponential^ 2). 


Sum of Independent Random Variables: 


Suppose X l , X 2 ,..., X n are n independent random variables, and the random variable Y is 
defined as 


Then, 


Y = X { + X 2 + 


+ X, 


n * 


My(s) = E[e sY ] 

= E[e s ^ x i +x 2 + +x «>] 

= E[e sX i e sX 2--e sX »\ 

= E[e sX \\E\e sX i\ • -E[e sX "] (since the X-s are independent) 
= M x (s)M x (,s)-M x (s). 


IfX 1? X 2 ,..., X n are n independent random variables, then 
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M x x +x 2 + ••• +x ,( s ) M x^ s ) M x 2 ^ s )‘" M xj s )- 


Example 6.8 

If A ~ Binomial(n,p ) find the MGF ofX. 

• Solution 

o We can solve this question directly using the definition of MGF, but an easier way 
to solve it is to use the fact that a binomial random variable can be considered as 
the sum of n independent and identically distributed (i.i.d.) Bernoulli random 
variables. Thus, we can write 

x=x l+ x 2 + -+x n , 

where X i ~ Bernoulli(p). Thus, 

M%(s) = M x (s)M x Is)-M x (s) 

= (M x (s)) n (since the X-s are i.i.d.) 

Also, 

M x (s) = E[e sX i] = pe s + 1 - p. 

Thus, we conclude 

M x ( s ) = (P e ' S + 1 ~pT- 


Example 6.9 

Using MGFs prove that ifX ~ Binomial(m, p) and Y ~ Binomial{n,p ) are independent, then 
X+ Y ~ Binomialim + n,p). 

• Solution 

o We have 

M/s) = (pe s + 1 ~p) m , 

My(s) = ( pe s + 1 ~ p )”. 

Since X and Y are independent, we conclude that 
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^ x+ >^' s ) M x (s)M y(.s) 

= (pe s +\-p) m+n , 

which is the MGF of a BinomiaKjn + n,p) random variable. Thus, 
X + Y ~ BinomiaKm + n,p). 


<— previous 
next —> 
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